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DETAILED ACTION 

1 . In view of the Appeal brief filed on 6 June 201 1 , PROSECUTION IS HEREBY 
REOPENED. New ground rejection is set forth below. 

To avoid abandonment of the application, appellant must exercise one of the 
following two options: 

(1 ) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply 
under 37 CFR 1 .1 13 (if this Office action is final); or, 

(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41 .31 followed 
by an appeal brief under 37 CFR 41 .37. The previously paid notice of appeal fee and 
appeal brief fee can be applied to the new appeal. If, however, the appeal fees set forth 
in 37 CFR 41 .20 have been increased since they were previously paid, then appellant 
must pay the difference between the increased fees and the amount previously paid. 

A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by 
signing below: 

/Samir A. Ahmed/ 

Supervisory Patent Examiner, Art Unit 2624 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
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invention was made to a person liaving ordinary sl<ill in tlie art to wliicli said subject matter pertains. 
Patentability sliall not be negatived by tine manner in wliicli tlie invention was made. 

3. Claims 1 -2, 4-6, 1 6-1 7, 1 9-20, 29-30 and 32-39 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Michael et al. (US 5,768,443) in view of Nonay et al. 
(US 6,618,494). 

a. Regarding claim 1 , Michael discloses a method of fitting acquired fiducial 
data to a set of fiduclals (FIGs. 6 and 9, "The inventor discloses a method of image 
distortion correction and local-to-global coordinate transformation using pixel and 
landmark" at col. 16, lines 4-12) on a fiducial plate ("semiconductor wafer" at col. 16, 
line 53) said method comprising: 

establishing a conversion from acquired coordinates to ideal fiducial coordinates 
using a data processing component (FIG. 6-48, "estimate camera distortion correction 
for each camera"; equations (1)-(5) and (13)-(39); "The inventor discloses fitting 

i+j<n i+j<n 

(computing) a function- x = G^(u,v)= J^a. a'v' and y = Gy(u,v)= ^b^ju'v' , to a set of 

Lj>0 !,,/>0 

data for the correction map." at col. 7, line 40 to col. 8, line 25 and col. 12, lines 22 to 
col. 15, line 40); 

calculating an absolute location for each identified acquired image feature 
centers ("pixels associated with feature points" at col. 16, lines 57-58) relative to the 
fiducial plate in fiducial plate coordinates (FIG. 9-56, "The inventor discloses that all of 
the pixels or the pixels associated with feature points are corrected for camera distortion 
using distortion correction map. The distortion correction map has a fitting function. 
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x = G^ (m, v) = ^^Uyu'v' and y = Gy (m, v) = ^byu'v' that can a point of an observed 

iJ^O i,j>0 

coordinate or an image coordinate {u,v} can be transformed to a model coordinate or a 
physical space {x,y}, which is a transformation or a calculation of a point or a pixel in 
one coordinate to the other one. Moreover, it well known that a coordinate or a 
coordinate system does tell the location of a pixel or a point from the center of the 
coordinate or coordinate system." at col. 7, line 40 to col. 8, line 25, col. 12, lines 22 to 
col. 15, line 40 and col. 16, lines 54-58) using the data processing component (The 
MPEP teaches that [t]he express, implicit, and /nA7erenf disclosures of a prior art 
reference may be relied upon in the rejection of claims under 35 U.S.C. 102 or 103^ 
[emphasis added]. So, "it is inherent that some sort of data processing component will 
be used for the camera corrected distortion using a distortion correction map because it 
uses coordinates of a pixel" at col. 7, line 40 to col. 8, line 25, col. 12, lines 22 to col. 15, 
line 40 and col. 16, lines 54-58) the absolute location indicating a distance 
measurement in fiducial plate coordinates ("It well known that a coordinate or a 
coordinate system does indicate the location of a pixel or a point from the center of the 
coordinate or coordinate system, which is an indication of the distance from the center" 
at col. 7, line 40 to col. 8, line 25, col. 1 2, lines 22 to col. 1 5, line 40 and col. 1 6, lines 
54-58); and 

based on at least one calculated absolute location of the identified acquired 

image feature centers (col. 16, lines 57-58, "pixels associated with feature points"), 
selectively modifying a structure represented by the identified acquired image feature 

^ See MPEP §2112 
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center ("It is disclosed that only pixels associated with feature points of interest are 
corrected" at col. 16, lines 57-58). 

However, Michael does not disclose fitting a fiducial grid model to data acquired 
by an imaging apparatus captured such that features are positioned in space relative to 

the fiducial plate. 

Nonay discloses fitting a fiducial grid model to data acquired by an imaging 
apparatus captured such that features are positioned in space relative to the fiducial 
plate (Fig. 1-405 and Fig. 13; "the determined grid points are preferably fitted to 
polynomials providing the best fit ..." at col. 16, lines 44-64). 

Michael and Nonay are combinable because both are related to the field of point 
transformation. 

At the time of the invention, it would have been obvious to a person of ordinary 

skill in the art to add the process of "fitting grid points" of Nonay to Michael's "method of 
image distortion correction and local-to-global coordinate transformation using pixel and 
landmark." 

The suggestion/motivation would have been to "remove optical distortion from 
digital image ... [and] quickly and efficiently correct optical distortion" (Nonay; col. 2, 

lines 51-58). 

b. Regarding claim 2, Michael further discloses wherein said fitting (FIG. 6- 
48, "estimate camera distortion correction for each camera"; equations (1)-(5) and (13)- 

(39); "The inventor discloses fitting (computing) a function- x = G^(u,v) = ^a^u'v' and 

ij>0 
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y = Gy (m,v) = ^byu'V , to a set of data for the correction map." at col. 7, line 40 to col. 

ij>0 

8, line 25 and col. 12, lines 22 to col. 15, line 40) comprises identifying fiducial 
coordinates for each fiducial captured in said data ("[T]he landmark feature of the 
calibration target is used to establish a local origin for the corrected physical coordinate 
system" at col. 7, lines 14-16) acquired by said imaging apparatus (FIG. 1-18, 20 or 22; 
col. 7, lines 41, "particular camera"). 

c. Regarding claim 4, Michael further discloses wherein said calculating 
(FIG. 9-56, "The inventor discloses that all of the pixels or the pixels associated with 
feature points are corrected for camera distortion using distortion correction map. The 

i+J<n 

distortion correction map has a fitting function, x = G^iu,v) = ^a^ju'v' and 

ij>0 

y = Gy (m,v) = ^fei,M'v' that can a point of an observed coordinate or an image 

coordinate {u,v} can be transformed to a model coordinate or a physical space {x,y}, 
which is a transformation or a calculation of a point or a pixel in one coordinate to the 
other one. Moreover, it well known that a coordinate or a coordinate system does tell 
the location of a pixel or a point from the center of the coordinate or coordinate system." 
at col. 7, line 40 to col. 8, line 25, col. 12, lines 22 to col. 15, line 40 and col. 16, lines 
54-58) comprises utilizing a linear least squares operation (equations (18)-(22); 
"Singular Value Decomposition technique to solve least-squares problems ... Ax=b" at 
col. 12, line 61 to col. 13, line 27). 
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d. Regarding claim 5, Nonay further discloses further comprising assuming 
that a rotation of said imaging apparatus relative to a fiducial grid is negligible (Fig. 1 - 
405 and Fig. 13; "The determined grid points are preferably fitted to polynomials 
providing the best fit, which shows that the rotation of an imaging apparatus was not 
considered" at col. 16, lines 44-64). 

e. Regarding claim 6, Michael further discloses wherein said imaging 
apparatus comprising a charge- coupled device camera ("image device [] such as a 
CCD (charge coupled device)" at col. 6, lines 40-41). 

f. Regarding claim 16, claim 16 recites a computer readable medium 
encoded with data and instructions, said data and said instruction causing an apparatus 
executing said instructions comprising steps equivalent to claims 1 . 

See rejection of claim 1 for further explanation. 

g. Regarding claim 17, Claims 17 recites a computer readable medium 
encoded with data and instructions, said data and said instruction causing an apparatus 
executing said instructions comprising steps equivalent to claims 2. 

The claim is analogous and corresponds to claim 2. See rejection of claim 2 for 
further explanation. 

h. Regarding claim 19, Claims 19 recites a computer readable medium 
encoded with data and instructions, said data and said instruction causing an apparatus 
executing said instructions comprising steps equivalent to claims 4. 

The claim is analogous and corresponds to claim 4. See rejection of claim 4 for 
further explanation. 
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i. Regarding claim 20, claim 20 is analogous and corresponds to claim 5. 
See rejection of claim 5 for further explanation. 

j. Regarding claim 29, Michael discloses a method of accurately identifying 
a location of a feature relative to a fiducial plate comprising: 

acquiring an image of an object with an imaging apparatus (FIG. 1 -1 8, 20 or 22; 
col. 7, lines 41 , "particular camera"), the image comprising data concerning the position 
of a plurality of fiducial marks on a fiducial plate and data concerning the position of a 
feature of the object, the image being acquired such that the data concerning the 
position of a plurality of fiducial marks on a fiducial plate and data concerning the 
position of a feature of the object is obtained simultaneously (FIG. 6-48, "estimate 
camera distortion correction for each camera"; equations (1)-(5) and (13)-(39); "The 

inventor discloses fitting (computing) a function- x = G^(u,v)= ^a^wV' and 

ij>0 

y = Gy (u,v) = ^byu'v' that can a point of a observed coordinate or image coordinate 

ij>0 

{u,v} can be transform to a model coordinate or a physical space {x,y} at col. 7, line 40 
to col. 8, line 25 and col. 12, lines 22 to col. 15, line 40); 

calculating an absolute location of a center of each of the plurality of fiducial 
marks in the acquired image relative to the fiducial plate in fiducial plate coordinates 
using the data processing component, the absolute location indicating a distance 
measurement in fiducial plate coordinates (FIG. 9-56, "The inventor discloses that all of 
the pixels or the pixels associated with feature points are corrected for camera distortion 
using distortion correction map. The distortion correction map has a fitting function. 
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x = G^ (m, v) = ^ttyu'V and y = Gy (m, v) = ^^yu'v' that can a point of an observed 



coordinate or an image coordinate {u,v} can be transformed to a model coordinate or a 
physical space {x,y}, which is a transformation or a calculation of a point or a pixel in 
one coordinate to the other one. Moreover, it well known that a coordinate or a 
coordinate system does tell the location of a pixel or a point from the center of the 
coordinate or coordinate system." at col. 7, line 40 to col. 8, line 25, col. 12, lines 22 to 
col. 15, line 40 and col. 16, lines 54-58); and, 

determining a position of a feature of the object in the acquired image and 
modifying the determined position based on at least one calculated absolute location of 
the plurality of fiducial marks in the acquired image (FIG. 9-56, "The inventor discloses 
that all of the pixels or the pixels associated with feature points are corrected for camera 
distortion using distortion correction map. The distortion correction map has a fitting 



function, x = G^(u,v) = ^ayuV and y = Gy(u,v) = ^byu'v' that can a point of an 



observed coordinate or an image coordinate {u,v} can be transformed to a model 
coordinate or a physical space {x,y}, which is a transformation or a calculation of a point 
or a pixel in one coordinate to the other one. Moreover, it well known that a coordinate 
or a coordinate system does tell the location of a pixel or a point from the center of the 
coordinate or coordinate system." at col. 7, line 40 to col. 8, line 25, col. 12, lines 22 to 
col. 15, line 40 and col. 16, lines 54-58). 

However, Michael does not disclose fitting a fiducial grid model to the image data 
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to establish a conversion from coordinates of the plurality of fiducial marks acquired 
from the image to coordinates of the plurality of fiducial marks on the fiducial plate using 
a data processing component. 

Instead of Michael, Nonay discloses fitting a fiducial grid model to the image data 
to establish a conversion from coordinates of the plurality of fiducial marks acquired 
from the image to coordinates of the plurality of fiducial marks on the fiducial plate using 
a data processing component (Fig. 1-405 and Fig. 13; "the determined grid points are 
preferably fitted to polynomials providing the best fit ..." at col. 16, lines 44-64). 

Michael and Nonay are combinable because both are related to the field of point 
transformation. 

At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to add the process of "fitting grid points" of Nonay to Michael's "method of 
image distortion correction and local-to-global coordinate transformation using pixel and 
landmark." 

The suggestion/motivation would have been to "remove optical distortion from 
digital image ... [and] quickly and efficiently correct optical distortion" (Nonay; col. 2, 

lines 51-58). 

k. Regarding claim 30, claim 30 is analogous and corresponds to claim 2. 
See rejection of claim 2 for further explanation. 

I. Regarding claim 32, claim 32 is analogous and corresponds to claim 4. 
See rejection of claim 4 for further explanation. 
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m. Regarding claim 33, claim 33 is analogous and corresponds to claim 5. 
See rejection of claim 5 for further explanation. 

n. Regarding claim 34, claim 34 is analogous and corresponds to claim 6. 
See rejection of claim 6 for further explanation. 

0. Regarding claim 36, Michael discloses wherein the object is part of a 
semiconductor probe card ("semiconductor wafer" at col. 16, line 53). 

p. Regarding claim 37, claim 37 is analogous and corresponds to claim 29. 
See rejection of claim 29 for further explanation. 

q. Regarding claim 38, Michael further discloses further comprising: 
interposing a substantially transparent substrate having a plurality of fiduclals formed 
therein between the imaging apparatus and the object ("semiconductor wafer" at col. 16, 
line 53). 

r. Regarding claim 39, Michael further discloses further comprising: 
acquiring a succession of images with an imaging apparatus, each of the succession of 
images including both the object and the plurality of fiducial marks (FIGs. 6 and 9, "The 
inventor discloses a method of image distortion correction and local-to-global coordinate 
transformation using pixel and landmark" at col. 16, lines 4-12). 

4. Claims 3, 18 and 31 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Michael et al. (US 5,768,443) in view of Nonay et al. (US 6,618,494), and further in 
view of Thompson (US 5,020,123). 
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a. Regarding claim 3, the combination of Michael and Nonay discloses ail the 
previous claim limitations including said identifying coordinates for each fiducial 
(Michael; "|T]he landmark feature of the calibration target is used to establish a local 
origin for the corrected physical coordinate system" at col. 7, lines 14-16) and said 
calculating an absolute location (Michael; FIG. 9-56, "The inventor discloses that all of 
the pixels or the pixels associated with feature points are corrected for camera distortion 
using distortion correction map. The distortion correction map has a fitting function, 



x = G^ (m, v) = ^ayu'v' and y = Gy (u,v) = ^byu'v' that can a point of an observed 



coordinate or an image coordinate {u,v} can be transformed to a model coordinate or a 
physical space {x,y}, which is a transformation or a calculation of a point or a pixel in 
one coordinate to the other one. Moreover, it well known that a coordinate or a 
coordinate system does tell the location of a pixel or a point from the center of the 
coordinate or coordinate system" at col. 7, line 40 to col. 8, line 25, col. 12, lines 22 to 
col. 15, line 40 and col. 16, lines 54-58) of identified acquired image feature centers 
("pixels associated with feature points" at col. 16, lines 57-58). 

However, the combination does not disclose a selectively iterating process. 

Instead of the combination, Thompson, the same field of endeavor of detecting 
image distortion, discloses selectively iterating process (FIGs. 2-205, 2-206, 2-207, "The 
invention shows an iterating process that fiducial marking are identified or compared 
with a predetermined tolerance" at col. 3, line 53 to col. 3, line 64). 

The combination and Thompson are combinable because both of them are 
related to the field of image distortion. 
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The combination contains a "base" process of identifying coordinates for eacli 
fiducial as "the landmark feature of the calibration target being used to establish a 
local origin for the corrected physical coordinate system" (IVIichael; col. 7, lines 14- 
1 6) and calculating an absolute location as the step of "all of the pixels or the pixels 
associated with feature points are corrected for camera distortion using 

distortion correction map comprising a fitting function, x = G^(u,v) = ^a^u'v' and 



y = Gy (u,v) = ^byu'v' that can a point of an observed coordinate or an image 



coordinate {u,v} can be transformed to a model coordinate or a physical space 
{x,y}, which is a transformation or a calculation of a point or a pixel in one 
coordinate to the other one (Michael; col. 7, line 40 to col. 8, line 25, col. 12, lines 22 
to col. 15, line 40 and col. 16, lines 54-58) of identified acquired image feature centers 
corresponding to the "pixels associated with feature points" (Michael; col. 16, lines 
57-58), which the claimed invention can be seen as an "improvement" in that 
selectively Iterating the process of the indentifying coordinates of fiducial and 
calculating the absolute location of the image feature centers [emphasis added]. 

Thompson contains a known technique of selectively iterating process 
corresponding to the "iterating process that fiducial marking are identified or 
compared with a predetermined tolerance" (Thompson; FIGs. 2-205, 2-206, 2-207; 
col. 3, line 53 to col. 3, line 64) [emphasis added]. 

One of ordinary skilled in the art would have been recognized that applying 
Thompson's known technique of the selectively iterating process (Thompson; FIGs. 2- 
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205, 2-206, 2-207, "The invention shows an iterating process that fiducial marking are 
identified or compared with a predetermined tolerance" at col. 3, line 53 to col. 3, line 
64) as applicable to the "base" process of the combination, which comprises 
identifying coordinates for each fiducial (Michael; "|T]he landmark feature of the 
calibration target is used to establish a local origin for the corrected physical coordinate 
system" at col. 7, lines 14-16) and calculating an absolute location (Michael; FIG. 9-56, 
"The inventor discloses that all of the pixels or the pixels associated with feature points 
are corrected for camera distortion using distortion correction map. The distortion 

i+j<n 

correction map has a fitting function, x = G^iu,v)= ^a^u'v' and 

ij>0 

i+j<n 

y = Gy (u,v) = ^bifU'v' that can a point of an observed coordinate or an image 

coordinate {u,v} can be transformed to a model coordinate or a physical space {x,y}, 
which is a transformation or a calculation of a point or a pixel in one coordinate to the 
other one" at col. 7, line 40 to col. 8, line 25, col. 12, lines 22 to col. 15, line 40 and col. 
16, lines 54-58) of identified acquired image feature centers (Michael;"pixels associated 
with feature points" at col. 16, lines 57-58) would have yielded predictable results of 
selectively iterating process for identifying the coordinates of each fiducial and 
calculating the absolute location of the image feature center, which results in an 
improved process such as "[providing] [a] robust handling of distortion and noise" 
(Thompson; col. 6, lines 56-57), "area identification ...accurately performed with any 
number of fiducial markings ... [and] realized even in situations in which fiducial 
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markings are missing or are unrecognizable" (Tliompson; col. 6, lines 59-63) [emphasis 
added]. 

Therefore, it would have been obvious to combine Michael, Nonay and 
Thompson to obtain the invention specified in claim 3. 

b. Regarding claim 18, Claims 18 recites a computer readable medium 
encoded with data and instructions, said data and said instruction causing an apparatus 
executing said instructions comprising steps equivalent to claims 3. 

The claim is analogous and corresponds to claim 3. See rejection of claim 3 for 
further explanation. 

c. Regarding claim 31 , claim 31 is analogous and corresponds to claim 3. 
See rejection of claim 3 for further explanation. 

5. Claims 7 and 35 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Michael et al. (US 5,768,443) in view of Nonay et al. (US 6,618,494), and further in view 
of Leonard et al. (US 7,034,272 B1). 

a. Regarding claim 7, the combination of Michael and Nonay discloses all the 
previous claim limitation including imaging apparatus (Michael; "image device [] such as 
a CCD (charge coupled device)" at col. 6, lines 40-41). 

However, the combination does not disclose said imaging device comprising a 
complementary metal-oxide semiconductor device. 

Instead of the combination, Leonard, the same field of endeavor of calibration of 
the data measurements using coordinates, discloses said imaging device comprising a 
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complementary metal-oxide semiconductor device (FIGs. 4-10 and 5-10; "CMOS 
camera" at col. 4, lines 45-49). 

The combination and Leonard are combinable because both of them are related 
to the field of calibration of the data measurements using coordinates. 

The combination contains an imaging device which differed from the claimed 
device by the substitution of the CCD camera (Micahel; col. 6, lines 40-41 ) with 
complementary metal-oxide semiconductor device [emphasis added]. Leonard 
discloses substituted device as a CIMOS camera (FIGs. 4-10 and 5-10; "CMOS 
camera " at col. 4, lines 45-49), and their functions were known in the art to acquire 
an Image of an object [emphasis added]. 

One of ordinary skilled in the art could have been substituted one known 
element for another, which is substituting the CIMOS camera of Leonard (FIGs. 4- 
10 and 5-10; "CMOS camera" at col. 4, lines 45-49) for the imaging device being a 
CCD camera of Michael (col. 6, lines 40-41), and the results of the substitution 
would have been predictable resulting in acquiring image with less power or power 
consumption, the capability of accessing the region of interest of the image by 
Integrating easily with other components as it is well-known [emphasis added]. 

Therefore, It would have been obvious to combine Michael, Nonay and Leonard 
to obtain the Invention specified in claim 7. 

b. Regarding claim 35, claim 35 is analogous and corresponds to claim 7. 
See rejection of claim 7 for further explanation. 
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Conclusion 

6. No claims are allowed. 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to JOHN LEE whose telephone number is (571)272-9554. 
The examiner can normally be reached on Monday - Friday (Alt.) 7:30 a.m. - 5:00 p.m.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Samir Ahmed can be reached on (571) 272-7413. The fax phone number 
for the organization where this application or proceeding is assigned is 571 -273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
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